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Abstract 

We study the performance of a DPCM system with a recursively indexed quantizer (RIQ) under 
various conditions, with first order Gauss-Markov and Laplace-Markov sources as inputs. We show that 
when the predictor is matched to the input, the proposed system performs at or close to the optimum 
entropy constrained DPCM system. We also show that the if we are willing to accept a 5% increase in 
the rate, the system is very forgiving of predictor mismatch. 


1 Introduction 

Differential pulse code modulation (DPCM) is often used to efficiently convert an analog source such as 
speech, music or images into a digital form for communication or storage. Its efficiency is due to the 
exploitation of the memory in a source by the use of a predictor, which estimates the present source sample 
to be encoded, based on the quantized previous source samples. The performance of DPCM depends on two 
factors 

1. How well the predictor exploits the source memory, i.e. , how closely it can estimate the actual source 
samples. 

2. How well the quantizer is matched to the prediction error (the quantizer input). 

In order to maximize the goodness of prediction, the predictor is usually chosen based on the statistical 
properties of a given source. However, many physical sources, such as those listed above, exhibit statistically 
varying local properties which are usually quite distinct from their global ones. If this is the case and DPCM 
happens to operate on a segment whose statistics differ from the global ones, it operates in a mismatched 
state, which results in additional degradation in the reproduction [1]. 

Various schemes have been devised to handle this mismatch between the source and the predictor. These 
involve some form of adaptation of the quantizer and/or the predictor, by which DPCM quickly follows up 
the changing statistics of the source and prevents overloading of the system. However, this quick response 
of DPCM is not without its cost: the adaptation requires more implementation and operational complexity. 

Matching the quantizer to the statistics of the prediction error is even more difficult, as the quantizer 
structure itself effects the statistical properties of the prediction error. One way to obtain the statistics of 
the prediction error process is through the use of an orthonormal expansion [2, 3, 4, 5]. This has been used to 
optimize the quantizer through an iterative procedure [4, 6, 7, 8]. However, under operational circumstances 
this might not be a viable option. 
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In this paper we study the performance of a DPCM system operating on a Gauss-Markov source and a 
Laplace-Markov source. The DPCM system considered here, called a recursively indexed DPCM, consists 
of a uniform quantizer with infinitely many output levels, a recursively indexed binary encoder, and a 
first-order linear predictor. The quantizer is designed simply by specifying its step-size, and the predictor 
is non-adaptive. The Gauss-Markov and Laplace-Markov sources are chosen because of their use in the 
modeling of physical sources. The goal is to observe the rate distortion performance of this system, as 
well as the performance degradation when the predictor and the source are mismatched. We compare the 
rate distortion performance of the proposed system to the optimum results in the literature [ 6 ], where the 
quantizer was optimized using the iterative procedure mentioned above. 

The simulation results show that the rate distortion performance of the proposed system achieves or comes 
very close to the optimum performance at all rates studied. In the case of mismatch the simulation results 
show that only a 5 percent increase in the rate allows a rather wide range of the predictor mismatch to 
first-order Gauss-Markov and Laplace-Markov sources. They agree with a result in [9] reported for a 2-level 
optimized DPCM that the predictor coefficient does not significantly effect the optimality of DPCM. But 
unlike in [ 9 ] it is observed that a lower predictor coefficient than the source correlation coefficient is better 
for the low rate case. 

In Section 2 a recursively indexed binary encoder is discussed. In Section 3 DPCM is briefly reviewed and 
the DPCM mismatch problem is posed. In Section 4 the performance of a recursively indexed DPCM is 
considered and numerical results are presented. Conclusions follow in section 5. 


2 A Recursively Indexed Binary Encoder 

The DPCM system discussed here uses a quantizer with infinitely many output levels. This requires binary 
encoding of a countably infinite alphabet, which poses obvious problems in design and operation. An obvious 
and reasonable approach is first to represent the input alphabet using only finite many symbols and then 
to encode these symbols either using a fixed- to-fixed or fixed-to-variable length encoding. A recursively 
indexed binary encoder is used for just this purpose. 

The recursively indexed binary encoder considered in this paper is a two stage binary encoder: recursive 
indexing followed by an optimum (the minimum average codeword length) symbol-to- variable length binary 
encoder for the output of recursive indexing. 

Recursive indexing is a mapping of a countable set to a collection of sequences of symbols from another set 
of finite size [10, 11]. Given a countable set A = {ao, ai, . . .} and a finite set B = { 6 o, , &m-i} of size 

M , the recursive indexing of A by B is a mapping / of A to the collection of all sequences of symbols from 
B such that 


/(a,) = ■ ..bM-ibr if «' = q(M - 1) + r (1) 

S ^ ✓ 

q times 

where q and r are the quotient and remainder of i when divided by M — 1. Set B is called the representation 
set. Defined as such, recursive indexing is a one-to-one mapping, a symbol-to- variable length, M- ary, prefix- 
free code and therefore uniquely and instantaneously decodable. 

Since the second stage of the recursively indexed binary encoder is an optimum symbol- to- variable length 
encoder (the Huffman algorithm is used to design such), the statistic of the representation symbols must 
be computed. For this purpose we first compute the number of representation symbols needed to de- 
scribe a typical source sequence X \ , X 2 , . . . X n of the length n from set A. Define pjt = Pr(X = at), 
then the number no of the occurrences of symbol bo is computed as follows. Observe that 60 occurs once 
whenever no, &M-u 0 ^- 2 , • • . , • • * occur. The number of times these symbols occur is given by 

npo» ttpAf — 1 > xxp2M — 2t • • • 1 rtpjfc(Af-i), . . . and so on. Therefore, 
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( 2 ) 


CO 

n 0 = n^Pk(M-i)- 

k=o 


In a similar manner the number n ; of the occurrences of symbol bj are found to be: 


OO 

n i = f °r j = 0, 1, • ■ • , M - 2. 

k—Q 

oo M — 1 

"iw-i = ^^2 ^2 kpjfc(A/-i)+j- 
*=0 j=0 


(3) 


From these it is seen that on the average the number of representation symbols needed for n source symbols 
is 


M-l 

j=0 


(4) 


Therefore, the average number of representation symbols to represent one source symbol is 


, M - 1 
j=o 


1 + 


KM-l 

n 


oo M-l 

= f + ^2 kPk{M~ i)+j 

k = 0 j= 0 


(5) 


It is convenient to define the above expression as the expansion factor, denoted e, of the recursive indexing 
I. It is the factor by which one source symbol is expanded by the recursive indexing. The relative frequency 
qj of representation symbol bj , is computed as follows: 


?;=-• 


( 6 ) 


An optimum symbol-to- variable length binary encoder after the recursive indexing takes one representation 
symbol at a time and produces the corresponding binary sequence from a set of variable length code words. 
It is designed for example using the Huffman algorithm. Then its rate Rri , the number of binary digits 
per representation symbol, is bounded by 


H(B) < Rri < H{B) + 1 


(7) 


symbol-to- variable length code by the Huffman algorithm it is observed that rate R is almost equal to H(B), 
the lower bound, when M is large. 

The overall rate of the recursively indexed binary encoder then is bounded 


eH(B) < R < eH(B) + e 
Note that e approximately equals 1 if M is large. 


( 8 ) 


3 DP CM with Recursively Indexed Binary Encoding 

3.1 Source and DPCM 

Let us consider the encoding by a DPCM system of a first-order Gauss-Markov process, 
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X k = pX k -i + W k (9) 

where Wk is an independent identically distributed Gaussian with mean zero and variance . The source 
correlation coefficient p is between —1 and 1. 

The DPCM system consists of a quantizer, a predictor and a binary encoder In a typical operational cycle 
the difference Z* between the source output X* and its prediction X* is quantized by quantizer Q yielding 
Q(Zjt), which in turn is binary encoded. The predictor considered in this paper is a first-order linear 
predictor and therefore it is given by bYk-\ for some constant 6. 

We will say that DPCM is matched to the source if the source correlation p equals the predictor coefficient 
6 and that it is mismatched otherwise. 

The binary encoder in ordinary DPCM is either a fixed- to- fixed length or a fixed- to- variable length binary 
encoder. In the former the binary encoder takes the index of the quantizer output level, produces its binary 
representation and sends the binary sequence through the channel. In the latter blocks of quantizer outputs 
are buffered and (usually) entropy-encoded. 

The performance of a DPCM system will be measured by distortion and rate. The distortion incurred is 
defined to be 


D = limsup-^- V' E((Xk - Yit) 1 2 }- (10) 

It is well-known that the error of the DPCM system is that incurred by the quantizer alone and nowhere 
else, i.e., X* — Yk = Zjt — Q(Z*) Hence the distortion can be rewritten as 

D = limsup— V E((Zk - Q(Zt)) 2 \- (11) 

N t[ 1 J 

The rate is defined to be the average number of binary digits used to transmit one source symbol. In case 
of a fixed-to-fixed length binary encoder is given by [log 2 N], where N is the number or quantizer output 
levels. In case of a fixed-to-variable length entropy encoder it is approximately H(Q), the entropy of the 
quantizer output process. 

3.2 Recursively Indexed DPCM 

The DPCM system considered in this paper is a recursively indexed DPCM system. It is different from an 
ordinary system in the following two ways: 

1. The quantizer Q is an infinite level uniform quantizer with the thresholds being the mid-points of 
output levels. 

2. The binary encoder is a recursively indexed fixed-to-variable length encoder. 

The quantizer with infinitely-many output levels uniformly spaced yields granular distortion only. The 
magnitude of the distortion is bounded by A/2. Therefore, no matter how large the input to the quantizer 
is, due to bad prediction, its output is at most A/2 different from its input. Since an unoverloaded quantized 
value is available to the predictor at the next prediction, the system can track the source output, thereby 
yielding lower prediction errors. Due to this quick response the system does not have catastrophic error 
propagation, which DPCM with a finite number of quantizer output levels has when a pathological source 
sequence is encountered. 
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As discussed in Section 2, a recursively indexed binary encoder is necessary because the quantizer used 
has infinitely many output levels. We note that the encoder is not necessarily an entropy-encoder for the 
quantized process. 

The distortion for this system is given by (11), while the rate is simply the entropy of the representation 
symbols multiplied by the expansion factor. 


4 Simulation Results 

To test this system first order Gauss-Markov and Laplace-Markov random number generators with cor- 
relation coefficient p were used. The Laplace-Markov process was defined as in [6]. The variance of the 
innovation sequence was chosen so as to get a source variance of unity. For each realization of the process 
100,000 samples were used. 

4.1 Rate-Distortion Performance 

To obtain the rate-distortion performance, the predictor was matched to the source correlation coefficient 
and the step-size was varied. Each step-size A generated a distortion-rate pair which was then plotted. The 
results were overlaid on the optimum results from [6]. The results for p — 0.8 are plotted in Figures 1 and 
2 . 

For the Gauss-Markov source the rate distortion performance of the proposed system achieves (or almost 
achieves) the optimum performance for the entire range of rates. This is true for both the experimentally 
obtained rates from [6] as well as the asymptotic results. Recall that for the optimum results the quantizer 
was designed using a relatively complex iterative procedure, while for the proposed system the quantizer 
was designed by simply specifying the step-size. 

For the Laplace-Markov source, the proposed scheme again performs as well as the optimum scheme for 
almost all rates as fair as the experimentally obtained results are concerned. However, the proposed scheme 
provides better results than the asymptotic results in [6]. We presume this is due to an error in the asymptotic 
results (or our interpretation of them!). 

Similar results were obtained for p— 0.5, and 0.2 for both sources. The results seem to indicate that the 
problem of matching the quantizer with the input statistics can be easily resolved by the use of a recursively 
indexed quantizer. The quantizer can be easily designed by simply specifying the step-size A, which in turn 
can be specified based on the distortion requirements. 

4.2 Performance Under Mismatch Conditions 

To investigate the effect of mismatch between the predictor and model coefficients we used three values for 
the spacing A of the uniform quantizer: 2.5<r x , 1.5<r r , 0.2cr x . These correspond to low, medium, and high 
resolution (rate) quantizations, respectively. For each spacing for uniform- quantization, the source sequence 
is applied to the recursively indexed DPCM system with various values for the predictor coefficient. The 
quantizer output sequence is fed to the recursively indexed binary encoder with M representation symbols. 
The value of M ranges from 5 to 31. The results are shown in Figure 3, where the horizontal and vertical 
axis are respectively the predictor coefficient and the product of the rate and distortion. 

For the Gauss-Markov source, for low rate (A = 2.5 a x ) the distortion is about 0.44. The best performance 
is obtained when the predictor coefficient is around 0.6, where the rate is 0.58 bits/sample. This value is far 
below the source correlation 0.8. Note that in [4], the best performance was reported around 0.815, slightly 
higher than the source correlation. If 5% increase in rate is allowed, then Figure 3 shows that the predictor 
coefficient can be anywhere between 0.4 and 0.8. 
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For medium rate (A = 1.5(7*) the distortion is about 0.186, while A 2 /12 is 0.1875. The best predictor 
coefficient is around 0.7. The rate for this value of A and b is 1.12 bits/sample. The value of 6 is again lower 
than the source correlation. Again a 5% increase in rate allows the predictor coefficient anywhere between 
0.48 and 0.88. For high rate (A = 0.2(7*) the distortion is about 0.00333, while A 2 /12 is 0.00333. The best 
predictor coefficient is around 0.8, where the rate is 4.2 bits/sample, and can range from 0.0 to 0.99 if 5% 
increase in rate is allowed. 

We observe that the best predictor coefficient moves from below closer to the source correlation as resolution 
increases. This may be because for larger values of A the quantization noise tends to be magnified when the 
predictor coefficient is larger 1 . Also we note that the distortion expression (A 2 /12) is quite accurate even 
for large A. Similar results are observed for the Laplace-Markov source. 


4.3 Effect of the Alphabet Size M 

Finally we look at the size of the representation alphabet on the rate. Note that the larger the representation 
alphabet is for <a certain value of A the less likely it is that the encoder will enter the recursive mode. This 
implies that given a value of A larger values of M will tend to lower the value of the expansion factor e 
closer to 1, thus lowering the rate. If this is a very strong effect, recursive indexing looses some of its charm, 
as the smaller alphabet size of the reproduction alphabet makes it more amenable to entropy coding. The 
recursively indexed DPCM system was simulated with representation alphabet sizes 7,9,11,13, and 15. The 
results for a Laplace-Markov source are shown in Figure 4. Note that at low rates (large values of A)there is 
no difference between these sizes. At higher rates, there is noticeable difference as we increase the alphabet 
size from 7 to 11. After that point there is very little improvement obtained in the rate when the alphabet 
size is increased. 


5 Conclusions 

Recursively indexed DPCM, which features an infinite level quantizer coded with a finite alphabet entropy 
coder has been shown to be an efficient encoder for first order Gauss-Markov and Laplace-Markov sources. 
The use of a recursively indexed quantizer in a standard DPCM system seems to provide a solution both to 
the problem of matching the quantizer to the prediction error statistics, and the problem of exactly matching 
the predictor to the source, at least for these simple sources. 
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